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We present results of our investigation into student understanding of the physical significance and 
utility of the Boltzmann factor in several simple models. We identify various justifications, both 
correct and incorrect, that students use when answering written questions that require application 
of the Boltzmann factor. Results from written data as well as teaching interviews suggest that many 
students can neither recognize situations in which the Boltzmann factor is applicable, nor articulate 
the physical significance of the Boltzmann factor as an expression for multiplicity, a fundamental 
quantity of statistical mechanics. The specific student difficulties seen in the written data led us to 
develop a guided-inquiry tutorial activity, centered around the derivation of the Boltzmann factor, 
for use in undergraduate statistical mechanics courses. We report on the development process 
of our tutorial, including data from teaching interviews and classroom observations on student 
discussions about the Boltzmann factor and its derivation during the tutorial development process. 

This additional information informed modifications that improved students’ abilities to complete 
the tutorial during the allowed class time without sacrificing the effectiveness as we have measured 
it. These data also show an increase in students’ appreciation of the origin and significance of 
the Boltzmann factor during the student discussions. Our findings provide evidence that working 
in groups to better understand the physical origins of the canonical probability distribution helps 
students gain a better understanding of when the Boltzmann factor is applicable and how to use it 
appropriately in answering relevant questions. 
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I. INTRODUCTION 

The study of student understanding of advanced topics 
is becoming increasingly prevalent in physics education 
research [IHIEl- Investigating upper-division undergrad¬ 
uate students provides a snapshot of the intellectual jour¬ 
ney from novice introductory student to expert physicist 
that may reveal key components of this transition m- 
Moreover, the National Research Council has recently 
emphasized the need for more study of advanced under¬ 
graduate education in many science disciplines [20]. As 
part of a broader study on student learning in thermal 
physics, we have investigated student understanding of 
the Boltzmann factor with the goal of developing instruc¬ 
tional strategies to improve that understanding. 

Statistical mechanics provides a mechanism for un¬ 
derstanding the emergence of macroscopic phenomena 
from the collective properties of individual microscopic 
systems; as such, it is a cornerstone of contemporary 
physics. However, due to its complexity and sophistica¬ 
tion, students do not typically encounter statistical me¬ 
chanics until late in their undergraduate (or even grad¬ 
uate) studies, and comparatively little research has been 
done to document student difficulties and successes in 
this field [131 0113 UHl 123122| • This work showed that 
even after instruction students often struggle to distin¬ 
guish microstates of a system from macrostates and to 
appropriately relate the two. The fundamental assump¬ 
tion of statistical mechanics states that all accessible mi¬ 
crostates of a system (microscopic arrangements of a sys¬ 


tem’s particles in phase space) are equally probable P5] . 
Microstates that share common macroscopic properties 
(system volume, internal energy, etc.) may be grouped 
into measurable macrostates. The probability of find¬ 
ing the system in a particular macrostate, AAi, is de¬ 
termined by the number of microstates corresponding to 
that macrostate, i.e., the multiplicity, uji, normalized by 
the total number of microstates: 

P(M.) = (1) 

3 

Much of the intellectual effort of statistical mechanics is 
spent defining the relevant properties of the microstates 
and macrostates and determining the multiplicity given 
the macroscopic properties of the system [53] . 

Loverude reports that many students have difficulty 
distinguishing microstates and macrostates in the context 
of binary systems m- In one question he asked students, 
after flipping six coins, if the probability of getting five 
heads was more than, less than, or equal to the prob¬ 
ability of getting six heads; about 20% of the students 
incorrectly stated that the probabilities were the same, 
often claiming, “all probabilities have equal occurrences,” 
which is true for microstates but not for macrostates m 
p. 190]. In another question, students had to compare 
the probabilities of a six-child family having two different 
sequences of boys and girls (GBGBBG vs. BGBBBB). 
Over one third of students incorrectly stated that the sec¬ 
ond sequence was less probable because families are more 
likely to have equal numbers of boys and girls rather than 
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only one girl out of six, thus connecting the probabilities 
of a macrostate (the relative number of boys and girls) 
to an individual microstate (a specific birth sequence). 

Loverude also provides evidence that students struggle 
to distinguish microstates from macrostates, especially 
in the context of interacting systems. In the context of 
Einstein’s model for a solid lattice structure, Loverude 
asked students to determine the most likely energy distri¬ 
bution between two lattices of different sizes m- About 
40% of students incorrectly stated that the most proba¬ 
ble macrostate is the one in which each solid has the same 
amount of energy and disregarded the number of oscilla¬ 
tors within each lattice. Loverude also reports that stu¬ 
dents often add the multiplicities of interacting Einstein 
solids to determine the total multiplicity rather than ap¬ 
propriately multiplying them [18j . 

A key aspect of equilibrium statistical mechanics is 
that, when dealing with large systems (~ 10^^ particles), 
the most likely state of the system is overwhelmingly the 
most probable. This result is due to the fact that the sta¬ 
tistical spread of the macrostate probability distribution 
tends to decrease as cr oc where a is the standard 

deviation and N is the number of particles in the sys¬ 
tem. When N is large, nearly 100% of all microstates 
exist within a range of macrostates that are virtually in¬ 
distinguishable from each other; i.e., within the limits of 
measurable uncertainty. This single most likely “system 
state” is the equilibrium state (with microscopic fluctu¬ 
ations) of the macroscopic thermodynamic system [25]. 

Mountcastle, Bucy, and Thompson studied students’ 
understanding of probability distributions by asking 
them to determine the most probable number of “heads” 
when flipping N coins as well the uncertainty in this value 
(reported as a ± Aa) [22| • About a third of students in¬ 
correctly indicated that the relative uncertainty remains 
constant as N increases, e.g., Aa/a = 15% for all cases, 
and about 20% stated that the uncertainty covers the en¬ 
tire range of possible values (the most probable result is 
A/2 ± A/2). However, students readily recognized that 
performing additional measurements would reduce the 
uncertainty of the mean; e.g., using more rain gauges 
to measure amount of rainfall |22| . Further investiga¬ 
tion showed that students have difficulty reconciling the 
“overwhelmingly probable” equilibrium state with calcu¬ 
lations and graphs showing that the probability of the 
single most likely macrostate actually decreases with in¬ 
creasing A: Pmax = A!/ [2^ (A^/2)! (A/2)!] for the bi¬ 
nomial distribution. Some students took this idea to 
the extreme on the coin toss question by stating that 
the most probable result of flipping 6 x 10^^ coins is 
3 X 10^^ ± 1 heads. The distinction between a single 
discrete macrostate and an equilibrium thermodynamic 
“state” (consisting of a range of virtually indistinguish¬ 
able macrostates) is subtle and requires careful attention 
by both students and instructors. These results, along 
with Loverude’s [18], provide the foundation for studies 
into students’ understanding of the statistical treatment 
of thermodynamic systems where states are defined by 


continuous (rather than discrete) quantities. 

The canonical probability distribution defined by the 
Boltzmann factor has been described as “the quintessen¬ 
tial expression of the statistical mechanical approach” 
[5^ p. 109] and “the most powerful tool in all of sta¬ 
tistical mechanics” [23l p. 220]. By knowing the possible 
microscopic energy eigenstates, one may deduce the ther¬ 
modynamic equilibrium properties of any system at con¬ 
stant temperature, including average internal energy, free 
energy, entropy, pressure, heat capacity, etc. This con¬ 
nection between microscopic and macroscopic properties 
is known to be difficult in multiple contexts in physics 
[m [n [HHso] as well as chemistry m- As the core of 
statistical mechanics is this micro-macro connection, this 
topic is an optimal context for an investigation of this na¬ 
ture. Our investigation of student understanding of the 
Boltzmann factor provides additional information about 
difficulties students have with this connection; these re¬ 
sults have implications for studies of more complex sys¬ 
tems and topics. 

In this paper we present results of our investigation 
into student understanding of the physical significance 
and utility of the Boltzmann factor in several simple 
models. We identify various justifications, both correct 
and incorrect, that students use when answering writ¬ 
ten questions that require application of the Boltzmann 
factor. Results from written data as well as teaching 
interviews suggest that many students can neither rec¬ 
ognize situations in which the Boltzmann factor is ap¬ 
plicable, nor articulate the physical significance of the 
Boltzmann factor as an expression for multiplicity, a fun¬ 
damental quantity of statistical mechanics. The specific 
student difficulties seen in the written data led us to de¬ 
velop a guided-inquiry tutorial activity, centered around 
the derivation of the Boltzmann factor, for use in an un¬ 
dergraduate statistical mechanics course. We report on 
the development process of our tutorial, including data 
from teaching interviews and classroom observations on 
student discussions about the Boltzmann factor and its 
derivation during the tutorial development process. This 
additional information informed modifications that im¬ 
proved students’ abilities to complete the tutorial during 
the allowed class time without sacrificing the effectiveness 
as we have measured it. Our findings provide evidence 
that working in groups to better understand the physi¬ 
cal origins of the canonical probability distribution helps 
students gain a better understanding of when the Boltz¬ 
mann factor is applicable and how to use it appropriately 
in answering relevant questions. 


II. THE PHYSICS OF THE BOLTZMANN 
FACTOR 

Before discussing our research on student understand¬ 
ing of the Boltzmann factor, it is useful to provide an 
overview of the physics (and mathematics) of the Boltz¬ 
mann factor and the canonical partition function. The 
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particular derivation of the Boltzmann factor and the 
canonical partition function through which students are 
guided in the tutorial is included in the Appendix. 

The underlying assumption of the canonical ensemble 
is that the thermodynamic system has a fixed equilibrium 
temperature, a fixed number of particles, and may ex¬ 
change energy with its surroundings. A standard model 
for the canonical ensemble is a very small system in equi¬ 
librium with a large thermal energy reservoir (free to ex¬ 
change energy but not particles, see Fig. [^. The Boltz¬ 
mann factor is a mathematical expression for the proba¬ 
bility that a system in equilibrium at a fixed temperature 
is in a particular energy state, 

P{^,) cx (2) 

where tpj denotes the microstate with a particular energy 
Ej, k is Boltzmann’s constant, and T is the temperature 
of the system [32] . The decaying exponential form of the 
Boltzmann factor results from an expression of the mul¬ 
tiplicity of the reservoir derived from Boltzmann’s equa¬ 
tion, 

S = kln{ui). (3) 

As the energy of the system decreases, the energy (and 
multiplicity) of the reservoir increases in such a way that 
the total probability increases (see the Appendix for full 
details). 

The canonical partition function {Z) is the result of 
the normalization constraint that the sum of probabilities 
[P('0j)] over all j must be unity: 

„-Ej/kT 

= = i (4) 

3 3 

Z = (5) 

i 

where Z depends on temperature, but is independent of 
the energy value |33|. One may also express the par¬ 
tition function in terms of the energy of a macrostate, 
E\ 

Z= J D{E)e-^^'^'^ dE, ( 6 ) 

All E 

where the density of states function, D{E), accounts for 
the degeneracy (or multiplicity) of the macrostate. In 
this way the canonical partition function is equally valid 
for systems with discrete energy microstates, as in Eq. 
([^, and those with continuous energy distributions, as 
in Eq. 0. 

The canonical partition function can be used to ex¬ 
press equilibrium (macroscopic) thermodynamic quanti¬ 
ties. For example, the Helmholtz free energy of a system 
may be written as a function of Z , 

(7) 



FIG. 1. Sample system for the Boltzmann factor instructional 
sequence. An isolated container of an ideal gas is separated 
into a small system (C) and a large reservoir (A). The label 
“C ” is used to avoid confusion with entropy. 


derivatives of E yield information about the system’s en¬ 
tropy, pressure, magnetization, and many other thermo¬ 
dynamic variables. Moreover, the average energy of a 
system, {E) , may be expressed as a derivative of the nat¬ 
ural logarithm of Z. Due to these connections, Schroeder 
refers to the canonical probability distribution as “the 
most useful formula in all of statistical mechanics” [231 
p. 223]. The canonical partition function and the Boltz¬ 
mann factor are cornerstones of statistical mechanics, 
and a thorough understanding of when and how they are 
useful (when examining an equilibrium system at con¬ 
stant temperature) is essential for study in the field. 


III. STUDENT UNDERSTANDING OF THE 
ORIGIN AND UTILITY OF THE BOLTZMANN 
FACTOR 

We gathered data in several different forms to study 
student understanding of the Boltzmann factor from mul¬ 
tiple perspectives. In one investigation, we gave students 
an ungraded written survey to determine whether or not 
they use the Boltzmann factor in an appropriate con¬ 
text. Additionally, we conducted teaching experiments 
with several students as well as classroom observations 
to assess their understanding of the physical origin and 
significance of the mathematical expression of the Boltz¬ 
mann factor. Our results indicate that students often 
do not use the Boltzmann factor in appropriate con¬ 
texts, instead using vague notions of lower energies hav¬ 
ing higher probabilities to make conclusions about the 
ratios of these probabilities. Moreover, we find that stu¬ 
dents may not recognize the physical significance of the 
Boltzmann factor, even after having memorized its math¬ 
ematical derivation. 


A. Student use of the Boltzmann factor in 
appropriate contexts 

One desired result of teaching students about the 
Boltzmann factor is that they will recognize applicable 
situations and use it appropriately to make claims about 
probabilities of the occupation of specific energy states. 
The probability ratios question (PRQ, shown in Fig. 
probes their ability to do this. The correct solution to 


F = -fcrin(Z); 
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Consider a particle (Particle A) in a system with three evenly spaced energy 

n = 3 — 

- O.lOeV 

levels, as seen in the figure at right. The probability that Particle A is in the 

n = 2 — 

- 0.05 eV 

energy level is P^(n). 

n = 1 — 

- 0.00 eV 

A. Is the ratio of the probabilities Pa (3) /PaC^) greater than, less than, or equal to the ratio of the probabilities 

Pa{‘F)!Pa{1-)1 Please explain your reasoning. 



B. Consider a second single particle. Particle B, that can also only n 

Particle A 

Particle B 

be in three states. The energies of the three states of each sys- 1 

0.00 eV 

-0.05 eV 

tern are listed in the table at right. Both systems are in equi- 2 

-1-0.05 eV 

0.00 eV 

librium with a reservoir at temperature T. Is the ratio of the 3 

-hO.lOeV 

-1-0.05 eV 

probabilities Pr(3)/Pr( 2) for Particle B qreater than, less than, or equal to the ratio of the probabilities 

Pa{3)!Pa{‘F) for Particle A? Please explain your reasoning. 




FIG. 2. Probability Ratios Question (PRQ): given as an ungraded survey before tutorial instruction. 


the PRQ requires students to recognize three pieces of 
information: 

• The probability of a single particle being in each 
of three energy states is proportional to the Boltz¬ 
mann factor for each state 

• A ratio of exponential functions is the exponential 
of the difference of their exponents 

• The differences in energies between adjacent states 
are the same for each particle {AEn,n-i = 0.05 eV) 

The first two items indicate that each ratio of probabil¬ 
ities is an exponential function of the energy difference 
between the two states. The third item reveals that both 
pairs of ratios in the PRQ are equal [31]. Students were 
also considered to have given a correct explanation to 
part B of the PRQ if they stated that the two ratios 
were equal because the only difference between the two 
particles is the energy of the ground state. 


1. Recognizing the need for the Boltzmann factor 

The PRQ was administered to students in an upper- 
division statistical mechanics course at a land-grant 
research university in the northeastern United States 
(School 1); data were collected from seven successive 
classes {N = 50). Students at School 1 are typically se¬ 
nior undergraduates who have competed studies in classi¬ 
cal mechanics, electrodynamics, and quantum mechanics. 
The PRQ was also administered once in a single-semester 
upper-division thermal physics course at a comprehensive 
public university in the western United States (School 2, 
N = 32). Students at School 2 are typically junior under¬ 
graduates who have completed studies in modern physics 
and classical mechanics. The PRQ was administered at 
both schools immediately before students participated in 


guided-inquiry activities regarding the Boltzmann fac¬ 
tor and the canonical partition function (our Boltzmann 
Factor tutorial, see Sec. IV). At School 1 the activities 
were used after lecture instruction, and the PRQ was 
given after lectures. At School 2 the activities were used 
in place of lecture instruction, and the PRQ was given 
before instruction to establish a baseline for students’ 
understanding before the tutorial. 


Student responses to the PRQ were coded in two ways: 
first by the response given (equal to, greater than, less 
than, or other), second by whether or not the Boltzmann 
factor was used. Figure[3^ shows the response frequencies 
for the entire seven-year data corpus from School I, and 
Fig.UJo shows the response frequencies from School 2. 
Green diagonal stripes indicate the students who used 
the Boltzmann factor or stated that the energy of the 
ground state was irrelevant (in part B) to obtain their 
chosen answers; these students are considered to have 
used correct explanations regardless of which answer they 
chose. [35] . 

We used a grounded theory approach to analyze stu¬ 
dents’ explanations for their responses; the entire data 
corpus was examined for common trends, yielding cate¬ 
gories defined by the data, and all data were reexamined 
to group them into those defined categories [331 [37|. One 
goal of our analysis was to focus on describing rather 
than interpreting students’ explanations while defining 
the categories. In this way our analysis stays as true 
to the data as possible by limiting researcher biases and 
interpretations. This is consistent with Heron’s identifi¬ 
cation of specific difficulties [31] . 

The data represented in Fig. [^ suggest two questions: 

1) What is the prevalence of invocation of the Boltzmann 
factor, regardless of the correctness of the response? and 

2) How do students justify their answers if they do not 
apply the Boltzmann factor? To answer the first of these 
questions, the data show four categories of responses: 
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FIG. 3. PRQ pretutorial results, a) School 1: after lecture instruction on the Boltzmann factor over seven years {N = 50), 
b) School 2: before any instruction on the Boltzmann factor in one year {N — 32). The green diagonal stripes indicate the 
students who used the Boltzmann factor or stated that the energy of the ground state was irrelevant (in part B) to obtain 
their chosen answers. Students in the “Other” column often provided no explicit answer or stated that there was not enough 
information to determine the answer. Only 24 students from School 1 and four students from School 2 used the Boltzmann 
factor on both parts. 


• Correct response (equal to) using the Boltzmann 
factor (or stating that the energy of the ground 
state was irrelevant in part B) 

• Correct response without using the Boltzmann fac¬ 
tor 

• Incorrect response using the Boltzmann factor 

• Incorrect response without using the Boltzmann 
factor 

This coding scheme enables highlighting of the num¬ 
ber of students who are and are not invoking the Boltz¬ 
mann factor to answer the PRQ. A natural question as¬ 
sociated with this coding scheme is, how might someone 
invoke the Boltzmann factor but arrive at an incorrect re¬ 
sponse? One route is to make a computational error. On 
the other hand, one could compare the wrong ratios, but 
do so correctly using the Boltzmann factor. Data also 
indicate that some students imposed degeneracy terms 
when using the Boltzmann factor to answer the PRQ. In 
coding responses, a student who wrote that probability 
is related to a decaying exponential of the energy was 
coded as using the Boltzmann factor independent of the 
final answer obtained. Using the Boltzmann factor and 
stating that the energy of the ground state is irrelevant 
were grouped together because both are correct physical 
justifications for concluding that the ratios of probabili¬ 
ties in part B of the PRQ are equal. 

Table H] shows the percentages of students who occupy 
each of the four response categories at each school for 
both parts of the PRQ. From the data shown in Fig. 


and Table H] it is clear that the distribution of responses is 
different at the two schools. A Fisher’s exact test showed 
this to be true {p = 0.008 for part A, p < 0.001 for part 
B) [3^41) . Another Fisher’s exact test showed that stu¬ 
dents at School 1 are using the Boltzmann factor on the 
PRQ pretest more than students at School 2 {p < 0.001 
on both parts) [12]. This is not surprising given that 
students at School 1 had received lecture instruction on 
the Boltzmann factor, while students at School 2 had not. 
On the other hand, only 48% of students at School 1 used 
the Boltzmann factor on part A and only 68% did so on 
part B, indicating that lecture instruction alone was not 
sufficient for all students to gain a robust understanding 
of when and how to use the Boltzmann factor. 

The most common incorrect response at 
School 1 for both parts of the PRQ is that 


TABLE I. Results from the PRQ pretest at both School 1 
{N = 50) and School 2 {N — 32). Students are grouped by 
whether or not they gave the correct answer (Equal to) and 
whether or not they used the Boltzmann factor (or stated that 
the energy of the ground state was irrelevant in part B). 


Part A Part B 

Correct Incorrect Total Correct Incorrect Total 


Used Bf 

34% 

14% 

48% 

62% 

6% 

68% 

No Bf 

6% 

46% 

52% 

6% 

26% 

32% 

Used Bf 

9% 

3% 

12% 

16% 

0% 

16% 

No Bf 

13% 

75% 

88% 

22% 

63% 

84% 
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P(0.10 eV)/P(0.05 eV) < P(0.05 eV)/P(0.00 eV) 

(“less than” for part A and “greater than” for part B; 
see Table 0. These answers are considered consistent 
because the second and third energy levels in particle B 
have the same numerical values as the first and second 
energy levels in particle A, respectively. A Fisher’s exact 
test shows the distribution of “less than” and “greater 
than” responses from School 1 to be significantly dif¬ 
ferent for part A as compared to part B (p = 0.035). 
However, the data from School 2 show the exact opposite 
trend: more students answer “greater than” for part A 
and “less than” for part B (see Fig. and Table ig. A 
Fisher’s exact test shows that this difference at School 
2 approaches significance (p = 0.061). Additional tests 
show that the results from School 2 are significantly 
different from those at School 1 (p = 0.038 for part A 
and p = 0.036 for part B). 

Figure^ also shows that students at School 1 are more 
likely to answer part B correctly (which discusses an ef¬ 
fective shift in the ground state energy of a system) than 
part A (comparing two different sets of probabilities for 
states within the same system), with only 34% using the 
Boltzmann factor to obtain the correct response on part 
A compared to 64% providing a correct explanation on 
part B (statistically signihcant, p = 0.035). One student 
at School 1 justihed his response for part B in stating 
that, “... it does not matter what the ‘baseline’ is, just 
the amount of energy added.” This higher performance 
on part B could be a result of our coding scheme in that 
explanations involving comments about the arbitrariness 
of the ground state energy were considered correct for 
part B regardless of the student’s response to part A. 
This phenomenon is not significantly observed at School 
2 (see Fig.|^, Fisher’s exact test yields p = 0.45). 


students at School 2 used the Boltzmann factor. Of the 
remaining students at each school, roughly half (15 out of 
26 at School 1 and 13 out of 28 at School 2) used a rank¬ 
ing of probabilities as their primary justification; e.g., 
Pa(1) > Pa{‘2) > Pa{5). An additional five students at 
School 2 stated that the lowest energy is most probable 
but did not make claims about the relative probabilities 
of energy states 2 and 3. Using probability ranking, ei¬ 
ther explicit or implied, is the most common incorrect 
justihcation at both schools, and no students provided 
a physical explanation for why the probabilities of the 
various energy levels would be ranked as they claimed. 

Of the students who ranked the probabilities to jus¬ 
tify their answers, eight students at School 1 and seven 
at School 2 made claims about the relative difference in 
probability between states 1 and 2 and between states 
2 and 3. Some claims were made in sentence form, 
e.g., “... it is more likely that the system will have 
less energy so the difference between [states] 3 & 2 is 
less than [between states] 2 and 1” (student’s emphasis); 
other claims took the form of a mathematical expression, 
e.g., “fA(l) — -Pa(2) > T’a(2) — Pa( 3).” Both of these 
statements imply the idea that Pa{^) ^ Pa{‘P) > Pa{5)- 
All seven students at School 2 used this idea to claim that 
Pa{?>)/Pa{‘^) > Pa{‘P)/Pa{^)- However, the students at 
School 1 used similar reasoning to come to three different 
conclusions: 


Pa{1) » Pa{2) > Pa{3) 
Pa(1) > -Pa(2) » Pa(3) 
^a(I) » -Pa(2) » Pa(3) 


Pa{^) . 

Pa{2) Pa(1)’ 

Pa{3) 

Pa{2) Pa(1)’ 

Ha (3) _ Ha (2) 
Pa{2) Pa{1)' 


( 8 ) 

(9) 

( 10 ) 


2. Incorrect reasoning about probability ratios 

The justifications students used to support their final 
answers were sorted into several categories developed us¬ 
ing a grounded theory approach. At School 1, 24 students 
(out of 50) used the Boltzmann factor within their expla¬ 
nation of their answers on the PRQ; only four out of 32 


TABLE II. Pretest Response Comparison: “greater than” vs. 
“less than.” Numbers shown indicate the percentage of incor¬ 
rect responses at each of the two schools. This is necessary 
because significantly more students answered the PRQ cor¬ 
rectly at School 1 than at School 2. Only by looking at the 
percentages of incorrect responses can meaningful compar¬ 
isons be made. 


Part A Part B 

greater than less than greater than less than 
School 1 23% 38% 44% 19% 

School 2 54% 25% 26% 44% 


Interestingly, this third case was used to justify a correct 
response. 

In each of these cases, students seem to be considering 
the probabilities in pairs and using the relative difference 
between each pair to compare the ratios of the pairs. This 
is consistent with a strategy for comparing fractions that 
Smith refers to as Compare Numerator-Denominator Dif¬ 
ferences (NDD) [43]. The NDD strategy is categorized by 
students using the within-fraction difference between the 
denominator and the numerator as a comparative mea¬ 
sure. Examples of the NDD strategy in Smith’s study 
include students determining that 3/5 = 5/7 (because 
5 — 3 = 7 — 5 = 2) and that 14/24 > 7/12 (because 
24—14 > 12 — 7) [13| . In our case, students are explicitly 
or implicitly using the differences between the probabil¬ 
ities as justihcation for comparing their ratios. Arons 
cites difficulties interpreting ratios as one of the most 
prevalent cognitive gaps for students at the secondary 
and undergraduate levels |44l p. 4-9]. 

Students who ranked the probabilities as simply 
Ha(1) > Pa{2) > Pa{3) also made claims that are con¬ 
sistent with some of Smith’s other classihcations. Using 
this ranking to claim that PA{i)/ Pa{2) > Pa{2) /Pa{P) 
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(“greater than” on part A) is consistent with Smith’s 
Denominator Principle (DP, fractions with larger de¬ 
nominators are smaller than fractions with smaller de¬ 
nominators), and using this ranking to claim that 
PA(3)/Py4(2) < Pyi(2)/P^(l) (“less than” on part A) 
is consistent with both the Numerator Principle (NP, 
fractions with larger numerators are larger) and Larger 
Components (LC, fractions with larger numerators and 
denominators are larger) |43j . However, since no student 
admitted to exclusively using either the numerator or the 
denominator of each ratio to compare the two, we cannot 
be certain that students used these strategies, only that 
the students’ final responses are consistent with their use. 

Students who used probability rankings to justify their 
answers were categorized as being consistent with one (or 
more) of Smith’s strategies. Reliability of the categories 
for our classification was checked by an independent clas¬ 
sification of the data from School 2. There was initial 
agreement for 72% of the student responses; after dis¬ 
cussion and negotiation, agreement of 91% and at least 
partial agreement of 97% of students was obtained (one 
analysis placed some students simultaneously in two cat¬ 
egories, while the other only agreed on one of the cate¬ 
gories for this group). 

At both School 1 and School 2 more student responses 
were aligned with the NDD strategy than either DP or 
NP/LC, and no significant differences were found be¬ 
tween the two student populations in terms of their use of 
these strategies. In many cases it is unclear precisely why 
a student chose the response s/he did based on the rank¬ 
ing provided, but it is interesting to note the similarities 
between their claims and those made by the adolescent 
students in Smith’s study. 

The key difficulty identihed so far is that many stu¬ 
dents do not apply the Boltzmann factor when it is ap¬ 
propriate to do so, even after lecture instruction. Instead, 
these students provide responses that are consistent with 
using novice-like reasoning strategies for comparing ra¬ 
tios. Most students recognized that lower energies are 
more probable, but they offered no physical justification 
for why this is so and couldn’t use this information alone 
to make conclusions about the probability ratios in ques¬ 
tion. 


IV), while the other two had not seen the tutorial. The 


interviews were conducted in the style of a teaching ex¬ 
periment HU 113 iS] and consisted of asking students 
to complete a guided-inquiry activity that started with 
asking them to consider how probability relates to multi¬ 
plicity in the divided container {C-R ) scenario (see Fig. 

and culminated with the derivation of the Boltzmann 
factor [17]. 

The teaching experiment is a unique form of inter¬ 
view as “it is an acceptable outcome... for students to 
modify their thinking” during the course of the in¬ 
terview HB]. According to Steffe and Thompson, “a 
teaching experiment involves a sequence of teaching 
episodes... [including] a teaching agent, one or more stu¬ 
dents, a witness of the teaching episodes, and a method 
of recording what transpires during the episode” [45] . 
For our purposes the interviewer alternated roles as both 
teaching agent and witness during each interview. In a 
sense, the activities used during the interview may also 
be seen as a teaching agent as they included tasks for 
students to complete, and students interacted with the 
document in an intellectual manner. Our goal for the 
interviews was not to simply determine students’ under¬ 
standing of the Boltzmann factor, but rather to examine 
how well they could complete instructional tasks based 
on previous knowledge related to the Boltzmann factor. 
Students worked on their own; the interviewer solicited 
explanations for their work and gave assistance when re¬ 
quired. Field notes were taken during the interviews, and 
students’ written work was collected afterward. 


TABLE III. Sample energy & multiplicity values for the “toy 
model” system (C) and reservoir (R; see Fig. [^. This table 
was presented to students during the teaching interviews and 
is also used in the Boltzmann Factor tutorial. A key element 
of this situation is that the combined energy of C and i? is a 
fixed value, Etot- 
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B. Recitation of a mathematical derivation withont 
physical understanding 

In an effort to probe student understanding of the 
Boltzmann factor more deeply, we conducted individual 
interviews with four students at School 1 after classroom 
instruction in the first year of tutorial implementation to 
determine their familiarity with the Boltzmann factor, 
its applications, and its origin. Two interview partici¬ 
pants had participated in the first half of the Boltzmann 
Factor tutorial during class (in which they discussed the 
definitions of macrostates, microstates, and multiplicity 
for the microcanonical and canonical ensembles, see Sec. 


Results from the teaching interviews provide further 
evidence of the need for the Boltzmann Factor tutorial, 
especially with regard to the origin of the Boltzmann 
factor itself. None of the interview participants found 
the tasks to be trivial, and none correctly articulated 
how the Boltzmann factor as an expression of probabil¬ 
ity relates to multiplicity prior to the interview. A major 
finding during these interviews was the identification of 
students’ difficulties in executing the Taylor series expan¬ 
sion as part of the derivation of the Boltzmann factor; we 
reported these difficulties previously m- 

One episode during one of the student interviews was 
of particular interest. One student (Joel [35] i '''^ho had 







participated in portions of the Boltzmann Factor tutorial 
in class) was very familiar with the applications of the 
Boltzmann factor and seemed to be just as familiar with 
its origin. In one portion of the activity, students were 
given a table of multiplicities for various discrete system 
energy levels and ask ed t o determine the most probable 
macrostate (see Table III I. The desired result was for stu¬ 


dents to conclude that the macrostate with the greatest 
reservoir multiplicity would be the most probable. Joel 
wanted to use the Boltzmann factor rather than think¬ 
ing about multiplicities, even though no information had 
been given about the relative energy values |49j . The in¬ 
terviewer asked Joel to show where the Boltzmann factor 
came from before applying it to this situation, at which 
point Joel quoted the textbook derivation of the Boltz¬ 
mann factor practically verbatim. The final portion of 
Baierlein’s mathematical derivation is as follows [26l p. 
92] [50], 


! \ _ + /multiplicity of reservoir when\ 

- const X J 


P{ipj) = const X exp 


^SniEtot - Ej) 


Piipj) = const X exp ^ 


SR{Etot) + 


OSr 


dpR 


(-E, 


Piipj) = (new constant) x eyzp {—Ej/kT). 


( 11 ) 

( 12 ) 

(13) 

(14) 


This derivation exploits the fact that the combined en¬ 
ergy of the system and reservoir (Etot) is a fixed quantity 
in order to write the energy of the reservoir (Er) in terms 
of the energy of the system {Ej). 

Joel’s ability to reproduce the derivation might sug¬ 
gest an understanding of the physical significance of the 
Boltzmann factor. However, when asked how the multi¬ 
plicity of the reservoir relates to the Boltzmann factor, 
Joel was at a loss. During his replication of the deriva¬ 
tion of the Boltzmann factor he had implicitly written 
that it was proportional to ujr, (connecting Eqs. 11 and 
14), but without explicit help from the interviewer, Joel 
could not recognize that the multiplicity of the reservoir 
when it has energy, Etot—Ej (RHS of Eq. ([TT])), is propor¬ 
tional to the exponential function, exp {—Ej/kT) (RHS 
of Eq. p^). Furthermore, Joel had great difficulty re¬ 
lating the physical example used in the textbook (a “bit 
of cerium magnesium nitrate... in good thermal contact 
with a relatively large copper disc” p. 91]) to the 
ideal gas example used during our interview. He was un¬ 
able to recognize and articulate the important physical 
characteristics of each scenario that make the Boltzmann 
factor applicable; e.g., a system with fixed temperature 
and variable energy. Joel’s failure to make these connec¬ 
tions suggests an incomplete understanding of the phys¬ 
ical reasoning used to derive the Boltzmann factor, even 
after memorizing the textbook derivation. 

Results from the teaching interviews and the PRQ sug¬ 
gest that many students can neither recognize situations 
in which the Boltzmann factor is applicable, nor artic¬ 
ulate the physical significance of the Boltzmann factor 


as an expression for multiplicity, one of the fundamen¬ 
tal quantities of statistical mechanics. These difficulties 
prompted our development of the Boltzmann Factor tu¬ 
torial to help students better understand the physical ori¬ 
gin of the Boltzmann factor and how it may be applied 
in various contexts. 


IV. DESIGN AND IMPLEMENTATION OF THE 
BOLTZMANN FACTOR TUTORIAL 

Given students’ apparent lack of recognition of when to 
apply the Boltzmann factor to a physical scenario, we de¬ 
signed a guided-inquiry tutorial activity to lead students 
through its derivation and encourage deep cognitive con¬ 
nections between the physical quantities involved. The 
derivation chosen for use in the Boltzmann Factor tuto¬ 
rial is included in the Appendix and may be found in 
many widely used textbooks, including the one used at 
the primary research site |26j . 

Our Boltzmann Factor tutorial gives students the op- 
portnnity to productively struggle with the connections 
between the mathematical formalism and the physical 
interpretations within the derivation of the Boltzmann 
factor [15]. Fostering physics-mathematics connections, 
such as gaining facility with taking limits and mak¬ 
ing approximations, as well as knowing when to take 
these steps, is an important and nontrivial component 
of upper-division courses as students transition from 
novices to experts in the field [TO] . 

The desired student outcomes during the tutorial are 
consistent with the concept of productive disciplinary en¬ 
gagement (PDE) [51j . The small group setting, with 
explicit instructions to discuss responses and reasoning 
with group mates, fosters engagement, which is evident 
through student-student discourse. Because disciplinary 
content is the core of the tutorial, most engagement in 
a tutorial constitutes disciplinary engagement. Finally, 
Engle and Conant define productive disciplinary engage¬ 
ment as episodes where student are making progress in 
their engagement with the content [^ . Evidence of this 
productivity includes students recognizing their confu¬ 
sion about a concept or making a new connection as a 
result of the interaction. Indeed, the tutorial pedagogy 
and the materials themselves provide a setting designed 
to foster PDE. The pedagogy used in the canonical set 
of tutorials, Tutorials in Introductory Physics, to address 
specific student difficulties, fully matches the parameters 
of PDE [S2][Sni p. hi]. Our primary goal is that students 
discuss topics in a way that helps them progress through 
the tutorial tasks while gaining a better understanding 
of those topics (discussing relevant concepts, synthesiz¬ 
ing information, engaging with the connections between 
the mathematics and the physics, etc.). While in some 
cases other, less time-consuming pedagogical approaches 
may also foster PDE and help students with specific dif¬ 
ficulties, in this case the depth of the content and the 
difficulty of the sequence of steps in the derivation of the 











9 


Boltzmann factor suggested that a tutorial would be the 
most effective way to achieve this. Given that a lecture 
on this topic typically uses an entire class period, we ex¬ 
pected the tutorial would occupy a full class period as 
well as some time outside of class. 


A. The Boltzmann Factor Tutorial 


The Boltzmann Factor tutorial begins by asking stu¬ 
dents to consider an isolated container of an ideal gas. 
They are guided to recognize that the container has a 
fixed internal energy {Etot) and that all accessible mi¬ 
crostates are equally probable. 

Once the properties of the contents of the isolated con¬ 
tainer have been established, the students are presented 
with a scenario in which the container of ideal gas is sep¬ 
arated into relatively small and large sections (see Fig. 
[^. The small system of interest (C) is said to be in 
thermal equilibrium with the large reservoir (i?), and 
the students are asked to compare the values of various 
thermodynamic properties of C to those of R to highlight 
the fact that the intensive properties (temperature, pres¬ 
sure) will have the same value for both C and R, while 
the values of the extensive properties (volume, number 
of particles, internal energy) of C are much smaller than 
those of R. 

The third section of the tutorial uses the fact 
that the multiplicities of C and R are so different 
{(jJc ^ Wfl) to justify a single-particle “toy model” in 
which ujc = 1 (wtot = = Wfl), and the en¬ 

ergy of C can only take on a handful of d iscrete values, 
Ec G {Ej} = {i?i, i? 2 , ■ • ■ } (see Table III). The students 


are asked to determine which system microstate (j = 1, 
2, 3, 4, or 5) is most probable and which is least proba¬ 
ble. The desired solution is that the system microstate 
in which the macrostate of R has the largest multiplic¬ 
ity {ojr) is the most probable (£'4 in Table III) because 


all reservoir microstates are equally likely. Careful con¬ 
sideration of the relative probabilities of each macrostate 
leads to the proportionality between the probability of 
the microstate of C and the multiplicity of the reser¬ 
voir: Piipj) oc LVrjlipj). 

The final section of the Boltzmann Factor tutorial is 
the derivation of the Boltzmann factor itself. The core 
of this derivation is a Taylor series expansion of Sfi{Efi) 
about the value En = Etot to obtain the exp ression for 
Sr as a linear function of Ec given in Eq. ( |A4[ ) [15] . The 
students are explicitly asked to consider the physical sig¬ 
nificance of each term in the expansion and to determine 
the final linear expression on their own. Then, using the 
relationship between entropy and multiplicity in Eq. (|^ , 
they are guided to derive an expression for ujr-. 


= gSfl/fc ^ ^Sn{Etat)/k-Ec/kT^ 


and because SR^Etot) is a constant, 
0 .^ oc 


(15) 


( 16 ) 


i.e., the Boltzmann factor. Students find that 
oc ujR{'tpj) and that u}R{i{jj) oc , leading 

to the proportionality in Eq. (H- Finally, they obtain 
the expression for the canonical partition function, Z, by 
normalizing the probability. 

The post-tutorial homework assignment is an appli¬ 
cation of the Boltzmann factor to a three-state system 
with unevenly spaced energy levels. Students are asked 
various questions about the ratios of probabilities of the 
system being in a particular state. These questions are 
similar to the PRQ but the students are given specific 
values for T and N and asked to determine numerical 
values for the probability ratio rather than compare two 
different ratios. They are also asked to determine an ex¬ 
pression for the generic ratio between the probabilities of 
any two energy levels. This homework assignment was 
used as a continuation of the tutorial, not as an assess¬ 
ment or research tool. 


B. Tutorial Implementation 


At School 1, the Boltzmann Factor tutorial was im¬ 
plemented after all lecture instruction on the Boltzmann 
factor. Students were given one 50-minute class period 
to complete the tutorial. The course instructor and one 
additional facilitator were available during the tutorial 
session as observers and facilitators [53] . No course credit 
is offered for participation in the tutorial itself, but the 
course grade does include a component for class partic¬ 
ipation. Several groups were videotaped during tutorial 
sessions (in three years of classes) to monitor tutorial 
progress and document student reasoning regarding the 
Boltzmann factor and related topics. 

The Boltzmann Factor tutorial was implemented at 
School 2 once in place of lecture instruction. Students 
were given one 50-minute class period and an additional 
20 minutes during the next period to complete the tu¬ 
torial. As described in Sec. Ill A[ the PRQ was admin¬ 
istered as a pretest at both institutions before tutorial 
instruction, and a similar question was used on course 
examinations. 


V. FINDINGS DURING IMPLEMENTATION 
OE THE BOLTZMANN FACTOR TUTORIAL 

An interesting question is whether recognizing when to 
use the Boltzmann factor serves as a direct proxy for an 
understanding of the physical significance and meaning of 
the expression — its origin and why it describes the rel¬ 
ative occupation of states. Instructors typically assume 
that this is the outcome of presenting the derivation of 
such functions to students: that the clear description of 
the steps of the derivation, including the explicit connec¬ 
tions between the mathematical steps and the physical 
constraints, assumptions, etc. that drive the mathemat¬ 
ics, provides students with the intended insight. Thus 
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the subsequent assumption is that the proper invocation 
of the Boltzmann factor implies an understanding of its 
meaning and significance. However, in the process of 
pilot-testing the Boltzmann Factor tutorial we observed 
that this is not necessarily the case. We have evidence 
from students working through sections of the tutorial 
either in class or during teaching interviews that suggest 
that (a) the students do not have a sense of the physical 
basis for the Boltzmann factor before the tutorial and (b) 
the sequencing of the tutorial provides the students the 
opportunity to gain an understanding and appreciation 
for this physical foundation. 

Below we describe the findings from the data collected 
during and after tutorial implementation. These data — 
collected in written and video form — provide evidence to 
support the claim that the activities the students work 
through in the tutorial improve both students’ ability 
to apply the Boltzmann factor appropriately and their 
understanding of the physical basis for the Boltzmann 
factor, including the connections between some of the 
mathematical steps and the physical scenario. 


A. Improving student use of the Boltzmann factor 
in appropriate contexts 

In order to probe the effect of tutorial instruction on 
student tendency to invoke the Boltzmann factor in an 
appropriate situation, we administered written post-tests 
at both schools on midterm examinations. The PRQ was 
given on a course examination after the Boltzmann Fac¬ 
tor tutorial in two years at School 1. A similar question, 
referred to as the PRQ Analog (shown in Fig. |^, was 
developed by the instructor at School 2 and asked on a 
course exam in one year at both institutions. The PRQ 
Analog requires students to apply the same knowledge 
as is used to correctly answer the PRQ: that the prob¬ 
ability of a particle being in one of the energy states is 
proportional to the Boltzmann factor, and that a ratio 
of probabilities would be equivalent to a ratio of Boltz¬ 
mann factors, which depends only on the difference be¬ 
tween the energy levels. However, the PRQ Analog adds 
some complexity by using systems with energy levels that 
are not evenly spaced and requiring students to recognize 
that the number of particles occupying each energy level 
will be proportional to the probability of a single particle 
having that energy. Despite this added complexity, Fig. 
[U shows that students’ use of the Boltzmann factor was 
very similar on both the PRQ and PRQ Analog exam 
questions. 

From the three implementations at School 1 there 
are 19 sets of matched (pre-/post-tutorial) data of stu¬ 
dents who participated in the Boltzmann Factor tutorial. 
There are 29 sets of matched data from School 2. Figure 

shows the exam data from all students broken down by 
question and school. These data provide evidence that 
the Boltzmann Factor tutorial helps students recognize 
the utility of the Boltzmann factor and how to apply it 


properly in the context of these questions. 

The most striking feature of Fig. is that all 13 stu¬ 
dents at School 1 used appropriate Boltzmann factor rea¬ 
soning on both parts of the PRQ m- Moreover, all but 
one student at School 1 (about 83%) used the Boltzmann 
factor correctly to answer the PRQ Analog after tutorial 
instruction. This is a marked improvement over lecture 
instruction alone (only about half consistently used the 
Boltzmann factor on the PRQ pretest). Similarly, all 
but two students at School 2 (also almost 95%) used the 
Boltzmann factor to answer the PRQ Analog exam ques¬ 
tion. 

In order to perform statistical analyses to compare the 
exam results with the pretest results, data were grouped 
into the four categories discussed in section |HIA1| This 
reduced coding scheme is necessary because we essen¬ 
tially asked three questions at various times (PRQ parts 
A and B, and the PRQ Analog): the specific responses to 
the various questions, e.g., “greater than,” cannot neces¬ 
sarily be considered the same response. As such, the only 
categories available for grouping responses are either the 
correct response or one of the incorrect responses; along 
with this we have the dimension of whether or not a stu¬ 
dent used the Boltzmann factor appropriately to justify 
his or her response, consistent with the four categories 
above. These general categories do not allow claims to 
be made about how reasoning patterns differ within the 
incorrect responses, but they do allow comparisons of 
the frequency with which students use the correct Boltz¬ 
mann factor reasoning and whether or not it yielded a 
correct response. Using these categories, a Fisher’s exact 
test showed that all exam data are statistically similar 
{p = 0.125). Additionally, a Fisher’s exact test compar¬ 
ing the data at School 1 showed that the results from 
the exams are statistically significantly better than the 
results on the PRQ pretest on both parts A {p = 0.019) 
and B {p = 0.012) [5^. A Fisher’s exact test also shows 
that the exam results at School 2 are significantly better 
than the pretest data {p < 0.001 for both parts). 

The written pretest results suggested that students 
were not aware of the contexts in which the Boltzmann 
factor is applicable, even after lecture instruction. The 
written post-test results demonstrate a marked improve¬ 
ment in the correct use of the Boltzmann factor in these 
situations. These results suggest that the Boltzmann 
Factor tutorial helps improve student understanding of 
how and when to use the Boltzmann factor when it is 
used either as a stand-alone activity (School 2) or as a 
supplement to lecture instruction (School 1). 


B. Improving student understanding of the 
physical basis for the Boltzmann factor 

As mentioned above, in addition to improving appro¬ 
priate student use of the Boltzmann factor, the other 
major goal for the Boltzmann Factor tutorial was for stu¬ 
dents to gain an appreciation for the physical basis of the 
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Systems A and B are both at the same temperature T. System 
A has N identical particles, each of which must be one of the 
three energy levels shown. In thermal equilibrium, the numbers 
of particles in the three levels are ni, n 2 , and n^. System B, 
with M identical particles, also has three energy levels, as shown. 
The numbers of particles in each of the three levels of system B 


System A System B 

{N Particles) (M Particles) 


3 - 0.10 eV 

2 - 0.06 eV 

1 - 0.00 eV 3 0.00 eV 


are mi, m 2 , and m 3 . 

2-0.06eV 

Which is the true statement? 1 - —O.lOeV 

I. The ratio 113/712 in system A is greater than the ratio m 2 /mi in system B. 

II. The ratio 713/712 in system A is equal to the ratio m 2 /mi in system B. 

III. The ratio 713/712 in system A is less than the ratio m 2 /mi in system B. 

IV. There’s not enough information to compare 713/712 in system A to m 2 /mi in system B. 

Explain your reasoning. If you answer IV, also say what additional information you would need. 


FIG. 4 


. PRQ Analog: developed by instructor at School 2. Administered on a course exam once at School 1 and at School 2. 



FIG. 5. Post-tutorial results from PRQ and PRQ Analog, 
administered during course examinations. The green diagonal 
stripes indicate the students who used the Boltzmann factor 
or stated that the energy of the ground state was irrelevant 
(in part B) to obtain their chosen answer(s). For the PRQ 
Analog, “Equal to” corresponds to choice II, “Greater than” 
to choice I, “Less than” to choice III, and “Other” to choice 
IV. Data are shown for students who completed the PRQ 
pretest, participated in the Boltzmann Factor tutorial, and 
completed either the PRQ or PRQ Analog exam question. 


Boltzmann factor, which did not occur based on lecture 
instruction, even when a student was able to recite the 
derivation exactly, as shown in section III B However, 
we anticipated that students would gain this apprecia¬ 
tion by working through the Boltzmann factor derivation 
in small groups while emphasizing the physical justifica¬ 


tions for each step therein. Documenting the acquisition 
of this appreciation or understanding is not possible us¬ 
ing written data of the sort typically gathered. So, in 
order to monitor student progress and success in achiev¬ 
ing this instructional goal for the Boltzmann Factor tu¬ 
torial, we videotaped several groups of students while 
they completed the tutorial during the first three years 
of implementation at School 1. 

Segments from these classroom episodes were selected 
for transcription and further analysis based on the con¬ 
tent of student discussions. Given our focus on investi¬ 
gating students’ understanding of particular topics, our 
methods of gathering video data align with Erickson’s 
description of manifest content approaches, in which par¬ 
ticular classroom sessions are selected to be videotaped 
based on the content being discussed m- We chose to 
videotape classroom sessions in which students were en¬ 
gaging in our tutorial because we are primarily interested 
in their ideas regarding the conceptual and mathematical 
content of our tutorial and students’ ability to negotiate 
tutorial prompts in an efficient and productive manner. 
We have already explained that our use of “productive” 
follows its use in productive disciplinary engagement EH; 
we classify “efficient” interactions as those enabling the 
students to complete the tutorial within the intended 50- 
minute class period. In some respects this categorization 
of student interactions is done with an eye toward the 
end justifying the means: an interaction cannot necessar¬ 
ily be considered productive or efficient without knowing 
the conversations that take place after that interaction. 

Over three years of tutorial implementation we video¬ 
taped a total of four groups containing 13 students. To 
analyze video data, we watched each video in its entirety 
and made note of conversations that seemed interest¬ 
ing; we later watched these segments many times and 
recorded both what was discussed and why we thought 


















































12 


it was interesting. Quotations included in this section 
were often selected for their uniqueness. Several students 
made comments and statements that indicated difficul¬ 
ties that were not expected and have not been previ¬ 
ously documented. Data do not exist to verify the per¬ 
vasiveness of these difficulties, but we feel their existence 
is noteworthy. In cases where more than one student 
displayed a similar difficulty, we have included multiple 
quotes to allow the reader to evaluate the similarities and 
differences between the data. 

Video data from the second tutorial implementation at 
School 1 provide evidence that students gain an appreci¬ 
ation for the origin of the Boltzmann factor while partic¬ 
ipating in the Boltzmann Factor tutorial. Two students 
(Sam and Bill, who worked in a group on their own) par¬ 
ticipated in several conversations throughout the tuto¬ 
rial session that indicate their contemplation of relevant 
physical ideas. During the Boltzmann Factor tutorial 
they discussed which macrostate (from Table III I is most 
probable: 


Bill - Probably the one with more microstates 

Sam - Yeah... the one with the highest 
multiplicity 

Bill - “Give a general expression for the prob¬ 
ability of the system”... so probably just 
use omega R (ujn), so we’d say omega R 
j ) over the sum of all of them. 

Sam - Yeah, that’s what we said: omega R j 
over the sum of omega R j {ujr^ / ^Rj ) • 


calculations. These data indicate that students who are 
able to successfully use the Boltzmann factor after lec¬ 
ture instruction may not have a complete understanding 
of the conceptual meaning behind the mathematics they 
are using. 

During the third year of tutorial implementation one 
group struggled to interpret the derivative of entropy 
(with respect to energy) obtained from completing the 
Taylor series as the inverse of the temperature T~^ (see 
Ref. [TS] for details). However, once they had written an 
expression for the entropy of the reservoir, one student 
had a particularly expressive realization upon solving for 
the multiplicity, 


“Actually wait, ohhh, heyyy, because then 
that becomes the partition [function]... and 
there’s your Boltzmann factor.” 


Similar statements were made by Jake(who had partici¬ 
pated in the first three sections of the tutorial in class) 
and others during the teaching interviews (see Sec. IIIB[ ) , 
indicating that they had not developed a robust under¬ 
standing of the physical significance of the Boltzmann 
factor after lecture instruction alone. All observation and 
interview data indicate that these same students can gain 
an appreciation for the physical significance of the Boltz¬ 
mann factor while participating in the Boltzmann Factor 
tutorial. 


C. Revising the tutorial 


Later in the tutorial, after completing the Taylor series 
expansion (with instructor intervention), interpreting the 
physical quantities involved, and relating their expression 
for multiplicity to the Taylor series of entropy, Sam and 
Bill had a realization [55] : 

Sam - That’s cool. Look, see, you get the Boltz¬ 
mann factor. You solve for omega (w): e 
to the minus E over k T (e“^/^^). 

Bill ~ I guess that’s where it comes from. 

Sam - ’Cause we didn’t know where it came 
from. 

Bill - I had no idea. 

Sam - I was just like, “OK.” 

These excerpts indicate that Sam and Bill are dis¬ 
cussing relevant physical quantities and principles and 
gaining an appreciation for the origin of the Boltzmann 
factor as a result of the Boltzmann Factor tutorial. In 
particular they are correctly relating the Boltzmann fac¬ 
tor of the system with the multiplicity of the reservoir 
as an indicator of probability. It should be noted that 
before tutorial instruction, Sam answered both parts of 
the PRQ correctly using correct reasoning, and Bill used 
the Boltzmann factor correctly but made errors in his 


The development process for instructional materials is 
iterative; modifications are typically made to improve 
the instructional experience based on earlier implemen- 
tation(s). For this reason data are collected during the 
tutorial implementations to ascertain the impact the ma¬ 
terials are having on students’ abilities to interact with 
the tutorial activities, including the extent to which: (a) 
students interpret the instructions as the developers in¬ 
tended; (b) the tasks and questions in the tutorial gen¬ 
erate productive discussions among the students, elicit 
specific difficulties targeted by the developers, and guide 
students to the desired outcomes; and (c) students are 
able to complete the tutorial tasks in the allotted time. 
The video data from School 1 serve this purpose, as does 
detailed feedback from the instructor at School 2 regard¬ 
ing students’ abilities to perform tutorial tasks as well as 
specific places where they had particular difficulty. Addi¬ 
tional data came from the teaching interviews described 
in section jill B[ which were conducted after the initial im¬ 
plementation at School 1 (when many students did not 
complete the tutorial). In this authentic instructional 
setting we find evidence of additional specific difficulties 
that written data would not elicit, as well as examples 
of student discussions prompted by the materials that 
inform the development of the tutorial and further re¬ 
search. 








13 


All of these data were used to inform tutorial revi¬ 
sions and modifications. Some revisions were minor, such 
as wording changes to improve clarity for the students. 
Other changes were more extensive, and included remov¬ 
ing sections entirely or moving activities and tasks to 
be completed either as pre-tutorial homework (the ini¬ 
tial steps in the Taylor series expansion of S[i{E[i) |T5] 1 
or post-tutorial homework (obtaining the expression for 
the canonical partition function). It should be noted that 
data do not exist to determine the precise effect that each 
individual tutorial modification has on student learning 
and understanding of the Boltzmann factor. However, 
the data do suggest that the collective modifications have 
led to increased student efficiency in completing tutorial 
tasks during later implementations, allowing students to 
complete more of the tutorial in the time allotted. In¬ 
creased efficiency benefits students by giving them the 
opportunity to arrive at the “punchline” of the Boltz¬ 
mann Factor tutorial: the derivation of the Boltzmann 
factor itself. 


During the first tutorial implementation at School I 
several unanticipated difficulties were observed. The first 
occurred while students completed the first page of the 
tutorial on which it asked them to “estimate (to order 
of magnitude) how many microstates (molecular config¬ 
urations) exist such that the total energy of the gas [in 
the isolated container] is Etot-" This language cued the 
students to attempt to find a formula for calculating the 
multiplicity of the gas based on its energy . The intent 
of the task, however, was for the students to recognize 
that there would be many many molecular configurations 
that would have a total energy of Etot and to just write 
down any appropriately large number. Students spent 
four minutes on this task before asking the instructor for 
help. (This wasn’t expected to take very long; a rigor¬ 
ous calculation was neither intended nor possible, and 
thus it should only have taken about a minute.) The 
wording of the question was altered in subsequent imple¬ 
mentations to ask the students, “How many microstates 
(molecular configurations) would you estimate exist such 
that the total energy of the gas is Etot- 1 , 1000 , 10 ^?” 
Data from the second tutorial implementation at School 
1 indicate that students found this order-of-magnitude 
estimate much easier than the year before. 


One observation noted during the teaching interviews 
was that some students focused strongly on a relation¬ 
ship between multiplicity and energy (w (x 
that was given in an introductory paragraph of the inter¬ 
view (and the tutorial section). The intent of the state¬ 
ment was to connect the Boltzmann Factor tutorial to the 
density of states function {D{E) = duj/dE oc E '^^/'^), 
which they had recently learned about, and to motivate 
the notion that luc ^ ojh (given that Vc Vr and 
Ec ^ Er). However, students tried to use this expres¬ 
sion to relate the multiplicities given in Table HI to the 
energies. One student (Jake, see p. 12) even stated that 
since the Ec = E^ microstate has the lowest reservoir 
multiplicity (ujr = 4 x 10^^, rightmost column in Table 


III), i ?3 must be the lowest energy (of C) and, therefore. 


be the most probable. What he failed to consider is that 
the multiplicity of the reservoir is the lowest, making Er 
the lowest, and E 3 the highest value (by conservation of 
energy). Jake’s reasoning, in fact, reached the exact op¬ 
posite conclusion of what was intended. 


The intent of the energy/multiplicity table (Table [lTT| ) 
and related questions is to motivate the connection be¬ 
tween multiplicity of the reservoir and probability of the 
system being in the corresponding microstate. The stu¬ 
dents were meant to realize that the Ec = A 4 microstate 
is the most probable since it has the largest correspond¬ 
ing multiplicity for the reservoir, leading them to con¬ 
clude that i 54 must be the lowest energy of the system 
because Er must be at its highest value. Two other 
interview participants displayed this tendency to latch 
onto the given expression relating multiplicity to energy; 
it was also observed during the in-class tutorial session to 
a lesser extent. The statement reminding students about 
the connection between multiplicity and energy was re¬ 
moved from later implementations of the Boltzmann Fac¬ 
tor tutorial along with most of the original introductory 
paragraph. 


Other in-class observations indicated that students did 
not always refer to their own work from previous sections 
of the tutorial when answering more difficult questions 
later. In particular, when answering questions about 
multiplicity concerning the divided container (see Fig.[^, 
students did not necessarily refer to the conclusions they 
had made about the original undivided container. Spe¬ 
cific references to previous tutorial sections were added to 
encourage students to make these connections and build 
on knowledge they had previously constructed. 

The most consistent observation from the first two 
implementations of the Boltzmann Factor tutorial at 
School 1 and the implementation at School 2 is that stu¬ 
dents could not complete the tutorial in one 50-minute 
class session. The students at School 1 during the first 
year were only able to complete the first three sec¬ 
tions of the tutorial, ending in an expression indicating 
Pi'ipj) oc They did not have the opportunity to 

even begin the Taylor series expansion that would lead to 
the derivation of the Boltzmann factor (the portion of the 
tutorial that we expected to be the most difficult). Af¬ 
ter revising the tutorial to address the specific difficulties 
discussed above, students at School I were able to suc¬ 
cessfully complete the first four sections of the tutorial 
(culminating with the derivation of the Boltzmann fac¬ 
tor) within one 50-minute class during the second year, 
but they were not able to complete the normalization of 
probability to determine an expression for the canoni¬ 
cal partition function. A similar result was reported at 
School 2 in that six out of seven groups of students (« 4 
students per group) were able to derive the Boltzmann 
factor after the entire 70 minutes allotted by the instruc¬ 
tor, but only 1-2 groups had enough time to derive Z as 
well. The students who did work through that portion 
of the tutorial, both those in class at School 2 and those 
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in the teaching interviews at School 1, had little trouble 
normalizing their expression for probability to get Z. 

Based on the overwhelming majority of students not 
completing the entire tutorial, even after modifications, 
we removed the fifth section of the tutorial, in which 
students derive the canonical partition function from the 
in-class activities, and added it as the first question in the 
post-tutorial homework assignment. The in-class portion 
of the tutorial now ends with the derivation of the Boltz¬ 
mann factor as well as a comment on the term “Boltz¬ 
mann factor” and a reference to the homework assign¬ 
ment in which students will determine an exact expres¬ 
sion for the probability rather than just a proportional¬ 
ity. Classroom observations from subsequent implemen¬ 
tations at School 1 indicate that these revisions have im¬ 
proved the efficiency of the tutorial, and that most stu¬ 
dents are able to complete the derivation of the Boltz¬ 
mann factor during a single 50-minute class period [60] . 


VI. SUMMARY AND IMPLICATIONS FOR 
FUTURE WORK 

Our results show that students often do not use the 
Boltzmann factor when answering questions related to 
probability in applicable physical situations after lecture 
instruction alone. These results have been replicated 
over several years. Students instead tend to use state¬ 
ments about a ranking of the relative probabilities to 
make novice-like claims about probability ratios, consis¬ 
tent with literature in mathematics education. This is a 
common error among students regardless of whether or 
not they had received lecture instruction on the Boltz¬ 
mann factor. To address students’ failure to appro¬ 
priately apply the Boltzmann factor, we developed the 
Boltzmann Factor tutorial to improve their understand¬ 
ing of situations in which the Boltzmann factor is appro¬ 
priate by guiding them through a derivation of the Boltz¬ 
mann factor, one that is particularly rich in connecting 
the physics to the progression through the derivation. 
Modifications were made to the tutorial based on teach¬ 
ing interviews and in-class observations in order to opti¬ 
mize student productive disciplinary engagement during 
class time. Results from several tutorial implementations 
indicate that students are far more likely to use the Boltz¬ 
mann factor properly after tutorial instruction than after 
lecture instruction alone (results are statistically signifi¬ 
cant at the p < 0.05 level). The Boltzmann Factor tuto¬ 
rial can be an effective supplement to (as at School 1) or 
replacement for (as at School 2) lecture instruction. 

We anticipated that guiding students through this par¬ 
ticular derivation of the Boltzmann factor would provide 
them with the opportunity to engage in the physical rea¬ 
soning behind the derivation of the Boltzmann factor, 
which our data suggested was not an outcome of lecture 
instruction — even for a student who invests the effort 
to memorize the textbook derivation. We have shown 
that participating in tutorial instruction on this deriva¬ 


tion helps students gain an appreciation of the physical 
implications and meaning of the mathematical formalism 
behind the formula that had previously eluded them; e.g., 
Sam and Bill. 

We have previously reported two related studies on 
students’ understanding of Taylor series expansions [15] 
and the relationship between the Boltzmann factor and 
the density of states as expressions of multiplicity m- 
These results support our current claim that deriving the 
Boltzmann factor is subtle and complex, and a robust 
understanding of its physical meaning is not trivial. 

One major avenue for future research is a study on 
the pervasiveness of student understanding of the Boltz¬ 
mann factor after tutorial instruction. Do students use 
the Boltzmann factor appropriately in situations that do 
not involve probability ratios of discrete, non-degenerate 
energy states? Do they recognize situations in which the 
Boltzmann factor is and is not applicable? Our original 
study has been necessarily focused on helping students 
understand a basic application of the Boltzmann factor. 
However, the Boltzmann factor is considered “the most 
powerful tool in all of statistical mechanics” [53] . Do stu¬ 
dents understand this tool well enough to use it to max¬ 
imum potential? Studying how students use the Boltz¬ 
mann factor and the canonical partition function to de¬ 
rive other physical quantities and investigating how well 
students understand the physical significance behind the 
relevant mathematical procedures could help answer this 
question and provide better insight into what students 
do and do not understand about the “quintessential ex¬ 
pression of the statistical mechanical approach” [26] . 
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Appendix A: Derivation of the Boltzmann factor 

To understand the mathematical form of the Boltz¬ 
mann factor, consider the interactions between the sys¬ 
tem under investigation (we call this C to avoid confusion 
with entropy, S ) and the thermal reservoir (i?; see Fig. 

[55]. The probability of finding the system in a par¬ 
ticular state will depend on the total multiplicity of the 
system-reservoir combination {P{Ec) oc ujtot), which is 
the product of the individual multiplicities of the system 
and the reservoir {cotot = In fact, if one con- 
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siders a small enough system (perhaps a single particle) 
the energy of the system may only occupy a handful of 
discrete energy levels {Eq € {Ej} = {Ei, E 2 , ••■})■ If 
these energy levels are non-degenerate, then the system 
would have a constant multiplicity, wc = 1 [13] ■ The to¬ 
tal multiplicity of the system-reservoir combination will 
then be exactly equal to the multiplicity of the reservoir: 

Wtot =0JRUJC = OJR. (Al) 


larger than the system, Er = Etot — Ec ~ Etot, and 
a Taylor series expansion is appropriate to approximate 
Sr{Er) about the point Er = Etot- 


Sr{Er) 


SR{Etot) 

SRiEtot) 


OSr 

OEr 


Ec + ... 


Ec 


(A4) 


The challenge now is to determine an expression for ujr in 
terms of Ec (the defining parameter of the macrostate). 
To accomplish this one must first relate Ec to the prop¬ 
erties of the reservoir. 

It is reasonable to assume that the system-reservoir 
combination is isolated from the rest of the universe such 
that its total energy, 

Etot = Ec + Er, (A2) 

remains constant. The energy of the system, however, 
may fluctuate about some average value, 

Ec = {Ec) ± 6 E. (A3) 

The magnitude of these energy fluctuations {5E) may 
be relatively large compared to {Ec), but insignificant 
compared to {Er), thus we are justified in considering 
R a reservoir as its energy does not change appreciably. 
Qualitatively, by conservation of energy, as the energy of 
the system decreases, the energy of the reservoir must 
increase, increasing lor and uttot, yielding a higher prob¬ 
ability; therefore, lower energy states for the system (C) 
are more probable than higher energy states. 

One must now be concerned with the precise mathe¬ 
matical form of multiplicity as it relates to energy, but 
while energy is an extensive variable, multiplicity is nei¬ 
ther extensive nor intensive. This dilemma is solved by 
relating multiplicity to the extensive quantity entropy via 
Eq. (§. Given that entropy is an extensive variable, it 
may also be written as a function of other extensive vari¬ 
ables; e.g., as Sr{Er). Because the reservoir is so much 


where (dS/dE) = T ^ from the fundamental ther- 

V,N 

modynamic relation (dE = T dS' — P dV + /i dN) and 
Er = Etot — Ec- The equality in the second line is valid 
because the temperature of the system (and reservoir) 
is fixed: higher-order derivatives of entropy are deriva¬ 
tives of temperature and thus vanish. In this manner 
one obtains an expression for Sr as a function of Ec and 
constants. Revisiting Eq. ^ one obtains, 

UJR oc .-. P{Ec) oc (A5) 

giving the desired result of P{Ec), from Eq. ([^. 

It should be noted that the above method is not the 
only way to derive the Boltzmann factor. Schroeder, for 
example, uses an approximation of the fundamental ther¬ 
modynamic relation rather than a Taylor series expansion 
to determine an expression for Sr in terms of Ec |23j . 
Carter, on the other hand, uses the method of Lagrange 
multipliers to maximize ln(a;) with the constraints that 
the average energy and number of particles in the sys¬ 
tem are both fixed; this derivation does not require the 
assumption of a large thermal reservoir, as the multiplic¬ 
ity of the reservoir is never used |64j . The derivation 
presented in this section was chosen for use within our 
Boltzmann Factor tutorial as it is presented in the text¬ 
book used at the primary research site [23] as well as 
several other commonly used texts (see Ref. [65] & [66] 1 
and because the physical significance of the Boltzmann 
factor (the multiplicity of the reservoir/surroundings) is 
emphasized throughout. 
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